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EVALUATION DESIGN REVIEW FORM 
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ESAA Administrator 
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PROGRAM SUMMARY 



On August 7, 1970, a lawsuit was filed in Federal District Court under 
authority of the Civil Rights Act of 1964 by the United States against 
the Texas Education Agency and seven school districts, including the 
Austin Independent School District. The complaint alleged that "the 
AISD was operating a dual school system based on race and was discrimi- 
natorily assigning Mexican-American students to schools identifiable as 
Mexican-American schools or as schools intended for only black and 
Mexican- American students" (Memorandum opinion and order, p. 1). In the 
memorandum opinion and order filed by United States District Judge Jack 
^ Roberts on November 5, 1979, the* Court found that the AISD f s past 
segregative actions have had systemvide impact as to the segregated 
Black housing patterns in Austin and the segregation of Black students 
in AISD schools, plus substantial iopact as to Hispanic students in 
East Austin. The Court; therefore, ordered the institution of a 
comprehensive desegregation remedy, including Hispanics in East Austin, 
to be in full operation by the beginning of the 1980-81 school year. 

Two primary goals, as specified by the Court, are taken from the publi^ned 
order: 

1. "... (To enhance) equal educational opportunity for students 
of all races and national origins, yet tempered by a realistic 
and genuine concern ft>r minimizing the disruptions of the lives 
of the individuals and families involved. . (; and — ) 

2. "... (To continue) making the decisions concerning school 
site locations, school capacity and construction, and school 
attendance zones with a view toward promoting maximum 
lasting integration" (pp. 50-51). 

The plan (known as "Plan A"), continues to be implemented by AISD. It 
Includes the following features: 

a. The establishment of a system of paired elementary school districts, 
so that one of the pair serves all children in grades oue through 
three for both schools while the other serves all children in grades 
four through six; 

b. An adjustment of the "feeder" pattern for district junior and 
senior high schools to achieve acceptable distributions of majority 
and minority students on the secondary level; 

c. The exclusion of all kindergarten children from reassignment outside 
qf their neighborhood elementary school districts; afid 

d. The discontinuance of the sixth-grade centers. 
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AISD also carries responsibility for program commitments to accompany the 
student reassignment plan. Some of xhese commitments include: 

a. The implementation of innovative programs designed to aid 
minority students, including^ilingual-bicultural education; 



b. The retenmon of the current majority-to-minor^ty transfer policy 
for studetffs with free transportation. * 



In the area of administration, faculty, and staff, AISD has made the 
commitment to continue affirmative action recruitment and employment 
efforts, including a survey of its divisions and departments to determine 
areas where minority representation needs improvement. 

In the area of construction and attendance zones, the District has made the 
following commitments, which include consultation with the Tri-ethnic 
Committee: \^ 

3* The District is to select sites and construct schools to prevent 
the ^recurrence of the dual school structure and to maximize 
integration; and 

b. The District is to alter pnd to draw attendance zones to promote 
desegregation. 
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EVALUATION SUMMARY 

The evaluation described in this design combines res6urces from the 
District's ESAA Out-of-Cycle grant with local funds' for the collection, 
analysis, and dissemination of information relevant to the Districtwide 
effects of the desegregation order described in the preceding section. 
This is the second year of the evaluation* As in the first year, this 
evaluation will focus on brqad questions relevant to the sytemwide impact ~~ 
of the desegregation order rather than on the specific ^activities funded 
from the ESAA grant. 

Student Achievement 

Again, the desegregation evaluation will assess the impact of. desegrega- 
tion on the achievement of AISD students* Do reassigned students achieve 
at the same J.evel as nonreassigned students? * 

The first year's findings indicated that nonreassigned minority students 
generally made greater achievement gains than their reassigned peers, 
while reassigned Anglo students scored as well cr better than nonreassigned 
Anglos* Undoubtedly, however, there were classrooms whei;e reassigned 
minority students achieved at higher levels than might have been expected. 
One pa^t of the evaluation will attempt to identify effective practices in 
those classrooms which might be used to ipprove the achievement of reassigned 
minority students in other classes. 

School Leavers * 

"White flight" to suburban and private schools in the wake of desegregation 
is a frequently addressed topic. One part of the evaluation will take a 
look at the more geneial problem of school leavers,, especially the dropouts 
and "pushouts." How many students leave the district annually? How many 
of those do not go to another school? Can students at high risk for 
dropping out be identified before they leave school? 

Faculty/Staff Recruitment Plan 

The consent decree requires the District to continue its efforts to meet 
the goals of the Faculty/Staff Recruitment Plan. * The final major area 
of the desegregation evaluation will be the evaluation of the Plan and 
the District's progress in meeting its goal. 
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i 

Much of the evaluation data used will be taken from already established 
' data sources within AISD such as the achievement files of Systemvide 
Evaluation, the Student Master File, and the Employee Master Record 
File. At the school level data collection may include informal class- 
room observations, interviews with school personnel, and the collection 
of information from school records. 

The staff assigned to carry out the evaluation consists of an evaluator, 
two half-time evaluation interns, an evaluation assistant, a programmer, 
and a clerk- typist. 
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DECISION QUESTIONS ADDRESSED 



Dl. Does the District need to make additional efforts to meet the 
achievement needs of students affected by desegregation? 

r* 

D2. ' Should t^e District invest in professional development to 

inform elementary teachers about classroom activities related 
to higher achievement among reassigned minority students 
(if such activities can be identified)? 

D3. Should the District provide additional attention to the 
identification of potential dropouts and to developing 
programs to keep them in school? 

V ft 

D4. Should the Austin Independent School District increase its 
efforts to pr^yide equal employment and equal promotional 
opportunities to all individuals? 



ERLC 6 u 



1MB 

DECISION QUESTIONS. OVERVIEW 



DECISION QUESTION 



/ 

Dl. Does the District need to make 
additional efforts to meet the 
achievement needs of students 
affected by desegregation ? 



DECfSION 
" DATE 



DATE 

NEEDED 
it 



RELEVANT EVALUATION 
QUESTIONS & OBJECTIVES 



Dl-1. What were the trends in 

achievement in AISU in 1981- 
1982? 

Dl-2. D'd students who were' reas- 
signed as a result of the 
desegregation process achieve 
at the same level as students 
in the, same schools who were 
not reassigned?. . .as students 
in schools which were not 
affected by desegregation? 

Dl-3. Were some schools more 

effective than others in 
boosting student achievement? 

Dl-4. Is there a relationship be- 
tween course selection by 
students (e.aCT the percentage 
of students Caking social 
studies classes) and the con-* 
tinuing decline in social 
studies achievement scores? 

Dl-5. Have there been changes In 
teacher attitudes and' 
practices during the second 
year of desegregation? 



INFORMATION SOURCES 



a) Sytemvide Testing Technical Report 



a) Iowa Tests of Basic Skills 

b) Sequential Tests of Educational 
Progress 



a) Iowa Tests of Basic Skills 

b) Sequential Tests of Educational 
Progress 



a) Seq, 
Proi 



tial Tests of Educational 

38 



a) Teacher Telephone interview 
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DECISION QUESTIONS OVERVIEW 



DECISION QUESTION 


DECISION 
DATE 


DATE 
NEEDED 


RELEVANT EVALUATION 
QUESTIONS & OBJECTIVES 


INFORMATION SOURCES j 


c 

D2. Should the District Invest In 
professional development to 
inform elementary teschers about 
classroom activities related to 
higher achievement anong re- 
assigned minority students (if 
such activities can be 
identified)? 






D2-1. Can elcaentsry classrooms be 
identified in whl^h reas- 
signed minority jtudenta made 
much lover and much higher 
than expected achievement 
gains in 1980-31? 

D2-". Do particularly effective an^ 
ineffective classrooms differ 
on such variables as size, 
percentage of low-income 
students , average achieve- 
ment level, etc*? 

D2-3. Do the teachers in particu- 
larly effective and inffec- 
tlve classrooms differ on 
such varlablea as age, number 
of years of experience, 
highest degree earned, etc.? 

D2-4. Can classroom actlvlt or 

practices be identifier which 
distinguish between the 
especially effective and in- 
effective classes? 

D2-5. Were there differences in 
the professional personnel 
evaluation ratings given to 
teachers in psrtlcularly 
effective classrooms compared 
to those received by teachers 
In ineffective classrooms? 


a) Iowa Tcata of Baaic Skills, 
a) Student Master File '» 

a) Employee Master Record Kile 

a) Informal Classroom Obscrvatlona 

b) Ter.cher Interview 

c) Principal r ntervlew 

a) Teacher Evaluation Form 
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DECISION QUESTIONS OVERVIEW 



DECISION QUESTION 



D3. Should the District provide addi- 
tional attention to the identifica- 
tion of potential dropouts and to 
developing programs to keep them In 
school? 



DECISION 
DATE 



DATE 
NEEDED 



RELEVANT EVALUATION 
QUESTIONS & OBJECTIVES 



D3-1. What procedures are followed 
ts> collect, record, and up- 
date enrollment data in 
AISD. 

D3-2. Wl'ut are the reasons for 
withdrawal given on the 
student master file? 

D3-3. Are there trends in the 

numbers of students leaving 
MSD in recent years?.... 
....in the reasons they 
leave? 

1)3-4." Can avnllable information be 
used to identify stu'ents 
wlto are likely to drop out 
of school? 

D3-5. Wlien a group of students is 
followed Tor several years, 
what do the findings reveal 
about: 

a) the number who drop out, 

b) the number who graduate, 

c) the number who drop out, 
then drop back in, 

d) the number who drop out 
during the summer com- 
pared with the number 
who drop out during the 
school year. 



INFORMATION SOURCES 



a) Interviews with District Employers 



a) Interviews with District Employers 



a) Student Mauter File 



a) Student Master File- 



a) Stux nt Master File 

b) School Records 
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DECISION QUESTIONS OVERVIEW 



DECISION QUESTION 



f>4. Should the Austin Independent School 
District increase Its efforts to 
provide equal employment and equal 
promotional opportunities to all 
Individuals? 



DECISION 
DATE 



December 1981 



DATE 
NEEDED 



December 1981 



RELEVANT EVALUATION 
QUESTIONS & OBJECTIVES 



CONTEXT 

D4-1. What arc the ethnic percent- 
ages of studejita in Texas 
and AISD? 

D4-2. What are the ethnic and sex 
percentages of teachers in 
Texas, AISD, and the nation? 

D4-3. What are the ethnic and sex 
percentages for administra- 
tors for the school year 
1980-81? 

D4-4. Wliat are the trends in 
employment by ethnicity 
over the years? 

D4-5. What an? the trends in the 
student population for AISD 
and Texas over the years? 

D4-6. How many teachers were certi- 
fied in 1980-81 by ethnicity 
and subject area? 

D4-7. How many student teachers by 
ethnicity were in the school 
district in 1980-81? 



INFORMATION SOURCES 



a) Student Master Kile 
h) Texas Education Agency 



a) Employee Master Record File 

b) Texas Education Agency 

c) Natlonsl Education Association 

a) Employee Master Record Kile 



a) -Employee Master Record Pile 

b) Office of Research and Evaluation 
Report (PuSU-atlon No; 80.59) 

a) Student Master 71 le 

b) Texas Education Agency 



a) T^xas Education Agency 



a) AISD Department of Staff Personnel 



p:v-? fu-H-y * 
waft risiiiLu 
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DECISION QUESTIONS OVERVIEW 



DECISION QUESTION 


DECISION 
DATE 


DATE 
NEEDED 


RELEVANT* EVALUATION 
QUESTIONS & OBJECTIVES 


INFORMATION SOURCES 










D4-8. How many students are in the 
» education department at U. T. 
by ethnicity and sex? 


a) 


College of Education (U.T.) 


• 








D4-9. How many teachers are leaving 
the district by ethnicity 
and sex? 


«) 


Employee Master Record File 






• 




RECRUITMENT AND IUR1NC 








« 

y 






D4-10. How many applicants (by 
ethnicity and sex) 
a) interviewed on caucus, 

1. completed application, 

2. were offered a 
position, 

3. were hired. 

b; interviewed in office, 

1* completed application, 

2. were offered a 
position, 

3. were hired. 

c) were hired again on a 
grant contract? 


«) 


Department of Staff Personnel 










D4-11. How many student teachers in 
the apecisl program (Dillard, 
Jarvis and Tan American) were 
offered positions nnd hired by 
ethnicity? 


n) 


Department of Staff Personnel 






s 




\ 
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DECISION QUESTIONS OVERVIEW 



DECISION QUESTION 



DECISION 
DATE 



DATE 
NEEDED 



RELEVANT EVALUATION 
QUESTIONS & OBJECTIVES 



D4-12. How tuny bilingual teachers 

a) were Interviewed, 

b) Bade application, 

c) were offered a position, 
and 

d) were hired? 

D4-13. How many alnorlty persons 
were hired into "Highly 
Visible positions?" 

D4-14. Were the 1980-81 goals met 
for AISD st the school and 
division levels? 



INFORMATION SOURCES 



a) Department of Staff Personnel 



a) Employee Master Record File 



a) Department of Staff Personnel 
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IV A 



INFORMATION NEEDS 



Deaegergation-related information needa by ESAA Program ataff and othera 
will undoubtedly ari8e. Their exact nature, however, cannot be anticipated. 
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INFORMATION NEEDS OVERVIEW 



INFORMATION NEED 

^ V I 


DATE 
NEEDED 


INFORMATION SOURCES 


Information needs cannot be specified in 
advance . 




< 
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DISSEMINATION 



/■ INFORMATION 


DISSEMINATION 
FORMAT 


DATE 


PERSONS 
RECEIVING v 


1980-81 Evaluation Findings 


Brochure and 
Oral Presenta- . 
tion 


September 
November 


ESAA Advisory 
Committee , 
UT Social Psycho- 
logy Department 
Students and 
Faculty 


1980-81 Evaluation Findings 

• • 


Brochure 


0 

October 

n 


AISD Principals 
and Teachers 
AISD PTA Presid- 
ents 

AISD Tri-Ethnic 
Committee 


1980-81 Evaluation Findings , 

* 


Article 


October 


Austin Alliance 
for a Smooth 
Transition News- 
letter 


1980-81 Evaluation Findings 


Paper Presenta- 
tion 


M^rch 


American Educa- 
tional Research 
Association 


Faculty /Staff Recruitment Plan 
Report 


Report 


January 


U.S. District ' 
Court 


Successful Desegregation Practices 


Brochure 


October 


AISD Principals 


Successful Reading Instruction 
Practices 


Brochure 


October 


AISD principals 


Self Study Questionnaires Based 
on School Effectiveness Research 


Questionnaires 


November 


AISD Principals 


1981-82 Evaluation Findings 


Final and 
Technical 
Reports 


June 


School Board, 
Administration, 
and Public 
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INFORMATION SOURCES 



INFORMATION 
| SOURCE 


POPULATION 


EVAi-.OUES.j 

referenced! 


1 1. Systemvtde Testing 
1 Technical Report 


All Students. 


Dl-l 


1 2. I aw it Testi of Basic 
1 JkillejttTBS) 


All students in grades 1-8. 


Dl-2, Di-3, D2-1 


1 3. Sequential! Tests of 
1 Educational Progcss 
1 (STEP) 


All students in grades 9-12. 


Dl-2, Dl-3, Dl-A 


1 4. Student Master Kile ^ 


All students within the 
DUtrlct. 


D2-2, D3-3, D3-A 
D3-5, DA-1, OA-5 


1 5. Employee Kaater Record 
1 Kile 


All teachers in the 
district. 

1 


nt-3, D4-2, DA- 3 
U4-4, DA-9, 
DA- 13 


1 6. Informal Classroom 
1 observations 


Selected** lasses. 

{ 


P2-A 


1 7. Teacher Interview 


A sample of elementary 
teachers in palted schools. 


D2-A 


1 ft. Principal Interview 


Seloi J principals 


D2-4 







1^81-82 Report 



ANALYSIS 
TECHNIQUES 



Descriptive Statistics 



Descriptive Statistics 
Analysis of Covariance 



Descriptive Statistics 
Analysis of Covariance 



February, 
March, 1981 



February, 
March, 1981 



February , 
March, 1982 



Descriptive Statistics 



Descriptive Statistics 



Content Analysis 



Content Analysis 



, REMARKS 



Demographic data about 
students - e.g., ethnicity, 
sex, grade, etc. 



Demographic data about 
teachers. 
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INFORMATION SOURCES 



INFORMATION 
SOURCE 


POPULATION 


EVALQUES. 
REFERENCED 


DATE 
COLLECTED 


ANALYSIS 
TECHNIQUES 


REMARKS 


9. Teacher Evaluation For» 


Selected teachers. 


D2-5 


1960-81 School 
Year 


Descriptive Statistics 
t- tests 




10. Interviews with District 
Employees 


Child accounting cleika, 
school registrars. 


D3-1, D3-2 


Fall, 1981 


Content Analysis 




11. Teacher Telephone 
Interview 


A sample of district elemen- 
tary and secondary teachers. 


Dl-5 


November, 1981 


Content Ansl>sls 




12. Texas Education Agency 


- 


D4-1, D4-2. D4-5, 
DA-6 


Fall. 1981 






13. 'School Records 


Records of school leavers. 


D3-5 


Pebruary t 
March, 1981 


Counts 




14. National Education 
Association 


All/ teachers In USA. 


D4-2 


Fall, 1981 


N/A 




15. ORE Report 80.59 


N/A 


DA-4 




N/A 


t 


16. A1SD Department of 
Staff Personnel 


All AISD teschers and 
student teachers. 


D4-7, DA-10, 
D4-11, D4-52, 
D4-14 


Fnll, 1981 


Count a 




17. College of Education 
(University of Texas) 


All education students at 
UT. 


D4-8 


Fall, 1981 


Counts 
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VII 



DATA TO BE COLLECTED IN THE SCHOOLS 



November, 
December, 1981 



February, 
March, 1982 



February, 
March, 1982 



February, 
March, 1982 



February, 
March, 1982 



A. Students 

B . Teachers 

1. Teacher Telephone Interview ; A telephone interview 
with a sample of District elementary. and secondary 
teachers • 

, 2. Teacher Interview ; Interview with a selected group 
of teachers concerning classroom-activities of 
reassigned minority students. 

3. Informal Classroom Observations ; Informal classroom 
observations in selected classes with reassigned 
minority students. 

C. Principals 

1. Principal Interview; Interview with principals 
concerning classroom activities with reassigned 
minority students. 

D. School Records 

1. Permanent Record Card, etc. ; An examination of 
records of school leavers. 
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VIII 

EVALUATION TIME RESOURCES ALLOCATION ■ 





ACTIVITY 


DIRECTOR 


EVALUATOR 


INTERN 


PROGRAMMER 


EVALUATION 


SECRETARY 




♦ 










ASSISTANT 




A. 


Design 


2 


5 


3 




3 


3 


E. 


Information Sources 
















1. Systemwide Testing Technical 


- 


- 


.5 


- 


- 


- 




Report. 
















2. Iowa Tests of Basic Skills 


1 


10 


A 


45 


12 


2 




3. Sequential Tests of Educational 


1 


10 


A 


33 


8 


2 




Progress 
















4. Student Master File 


1 


20 


10 


55 


20 


2 




5. Employee Master Record File 


- 


1 


2 


- 


- 


- 




6. Informal Classroom Observations 


.5 


3 


20 


- 


10 


10 




7. Teacher Interview 


.5 


A 


20 


- 


10 


1 




8. Principal Interview 


.5 


A 


20 


— 


10 


1 




9. Teacher Evaluation Form 


- 


.5' 


- 


5 


- 


- 




10. Interview of District Employees 




1 


8 




2 






11. Teacher Telephone Interview 


.25 


.5 




_ 


20 


5 




12. Texcs. Education Agency 




.25 


.5 






.25 




13. National Education Association 


- 




.25 


- 


- 






14. ORE Report 80.59 
















15. AISD Department of Staff Personnel 


- 


3 


6 


- 


- 


- 




16. College of Education, University 


— 


- 


.25 


- 


- 


— 




of Texas ^ 














C. 


Interim Dissemination 
















1- PindinttS Ri*rtphiiT*p 




A 
*♦ 






in 


im 




2. Findings Article 




.25 






.5 


.25 




3. Faculty/Staff Recruitment Plan 


1 


A 


11 




5 


3.5 


9 


Report 
















A. School Effectiveness 


.25 


.5 


A 




.5 


1 




Questionnaires. 































VIII 

EVALUATION TIME RESOURCES ALLOCATION 



ACTIVITY 


DIRECTOR 


EVALUATOR 


INTERN 


PROGRAMMER 


EVALUATION 
ASSISTANT 


SECRETARY 


5* Successful Desegregation Brochure 
6* Successful Instruction Brochure 

7. District Records Documentation 

8. Literature Summaries 


.25 
. 25 
.5 
.5 


1 
1 
2 


4 
4 
4 
20 


— 


2 
2 


1.5 
1. 5 
3 
5 


D. 


Ad Hoc Analysis 


C 


40 


70 


45 


40 


10 


E. 


Final and Technical Reports 


10 


60 


30 


- 


60 


65 


F. 


Other Dissemination 


5 


5 


5 




10 


15 


G. 


Administrative and Other Indirect 
Costs 


20 


50 


11 


5 


5 


60 


H. 


Total 


49. 75 


230 


Oil c 

211. 5 


188 


230 
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